fix(solver): reject non-finite input traces at the FFI boundaries#161
Merged
Conversation
A NaN/Inf in an input trace previously propagated silently: total_cmp sorts NaN last (corrupting the rolling-baseline percentile), alpha/PVE come back NaN, and the result is returned with converged=false — indistinguishable from a legitimately hard trace. In the batch path a NaN actually panicked deep in the FFT (Result::unwrap on a non-real spectrum), aborting the call. Add a shared crate::first_nonfinite helper and guard every FFI trace entry, returning a clear typed error (fail loud, no garbage): PyO3 (py_api.rs): - to_f32_vec (covers deconvolve_single, py_indeca_solve_trace, py_seed_trace, py_indeca_estimate_kernel, py_indeca_fit_biexponential) - PySolver::set_trace - deconvolve_batch (inline 2D row conversion bypasses to_f32_vec) - seed_kernel_estimate (inline 2D flat-buffer build) WASM (js_indeca.rs): indeca_solve_trace and seed_trace now return Result<JsValue, JsError> and throw on non-finite input. Both CaDecon worker handlers already wrap these in try/catch, so the error surfaces as a job error. The shared Solver core signature is unchanged (native/wasm/pyo3 all compile); validation lives at the entry boundaries. CaTune uses Solver.set_trace directly in the browser — noted as a follow-up (needs a shared-signature change). Tests: Rust first_nonfinite unit tests; Python test_input_validation covers single/batch run_deconvolution, seed_kernel_estimate, and PySolver.set_trace. The batch and seed_kernel_estimate gaps were caught by these tests, not by the 1D guard alone. All suites green (128 Rust, 6 Python, 289 TS). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Neither FFI boundary validated trace finiteness. A NaN/Inf in an input trace propagated silently:
total_cmpsorts NaN last, corrupting the rolling-baseline percentile pick;alpha/PVEcome back NaN, and the result is returned withconverged=false— indistinguishable from a legitimately hard trace;Result::unwrapon a non-real spectrum), aborting the whole call.This is a pre-publication defensibility gap: silent garbage results.
Fix
Add a shared
crate::first_nonfinite(&[f32]) -> Option<usize>helper and guard every FFI trace entry, returning a clear typed error (fail loud — chosen over sanitize/skip):PyO3 (
py_api.rs)to_f32_vec— coversdeconvolve_single,py_indeca_solve_trace,py_seed_trace,py_indeca_estimate_kernel,py_indeca_fit_biexponentialPySolver::set_tracedeconvolve_batch— inline 2D row conversion bypassesto_f32_vecseed_kernel_estimate— inline 2D flat-buffer buildWASM (
js_indeca.rs) —indeca_solve_traceandseed_tracenow returnResult<JsValue, JsError>and throw on non-finite input. Both CaDecon worker handlers already wrap these intry/catch, so the error surfaces as a job error (no silent garbage, no hang).The shared
Solvercore signature is unchanged (native/wasm/pyo3 all compile); validation lives at the entry boundaries.Tests
first_nonfiniteunit tests (clean / NaN / ±Inf).test_input_validation.pycovers single + batchrun_deconvolution,seed_kernel_estimate, andPySolver.set_trace.seed_kernel_estimategaps were caught by these tests, not by the 1D guard alone — both inline 2D paths bypassedto_f32_vec.All suites green: 128 Rust, 6 Python validation, 289 TS;
cargo fmt/clippyclean on both feature sets; WASM builds; typecheck clean.Follow-up (noted, not in this PR)
CaTune calls
Solver.set_tracedirectly in the browser — guarding that path needs a shared-Solver-signature change (it's used by native/pyo3/wasm/tests), so it's deferred to a CaTune-scoped change. CaDecon (the app under review) and all of Python are fully covered here.Part of the config/numerical-hygiene series (item: FFI NaN guard). Next: surface the biexp/degenerate fallback status.
🤖 Generated with Claude Code